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New  Tools  for  Hybrid  Systems 

Munther  A.  Dahleh^  *  Alexandre  Megretski^  1 
May  2,  2007 


Abstract 

We  provide  a  final  report  on  the  research  supported  under  AFOSR  #  FA9550-04-1- 
0052. 

1  Robust  Control  Framework  for  Hybrid  Systems 

Our  interest  in  deterministic  finite  state  machines  stems  from  our  ongoing  attempts  to 
use  these  systems  as  approximations  of  dynamical  systems  having  finite  input  and  output 
alphabets  [7] .  The  ultimate  goal  of  our  research  is  to  find  a  procedure  that  would  enable  us 
to  generate,  for  a  class  of  systems,  nominal  finite  state  machine  models  with  guaranteed  error 
bounds  that  are  usable  for  robust  controller  synthesis  [4].  The  issue  of  stability  comes  up 
in  several  ways  in  the  context  of  this  problem:  For  instance,  Hankel  model  order  reduction 
is  applicable  to  stable  LTI  systems,  is  guaranteed  to  generate  stable  reduced  order  systems, 
and  additionally  provides  a  quantitative  measure  of  the  quality  of  the  approximation  in  the 
form  of  induced  gain  bounds  that  are  compatible  with  robustness  analysis.  Similarly  for  our 
problem,  it  makes  sense  to  ask  if  there  is  some  notion  of  stability  that  guarantees  existence 
of  good  finite  state  machine  approximations,  and  to  ask  whether  the  approximate  models 
should  preserve  this  notion  of  stability.  It  also  makes  sense  to  look  for  meaningful  induced 
gain  conditions  to  quantify  the  approximation  error  and  to  quantify  the  sensitivity  of  our 
model,  and  hence  of  the  synthesized  closed  loop  system,  to  perturbations  due  to  unmodeled 
dynamics. 

Various  notions  of  stability  have  been  explored  for  a  range  of  discrete  event  and  finite 
state  systems.  Stability  in  the  sense  of  Lyapunov  and  asymptotic  stability  for  logical  dis¬ 
crete  event  systems  were  studied  in  [6],  The  problem  of  output  stabilization  of  discrete 
event  systems  was  considered  in  [5],  where  stability  was  understood  to  mean  that  all  state 
trajectories  of  the  closed  loop  system  pass  through  a  given  set  E  of  interest  infinitely  often. 
Questions  of  stability  of  equilibria  and  attractiveness  of  invariant  sets  were  addressed  in 
[3].  Extensions  of  LaSalle’s  Invariance  Theorem  and  Lyapunov’s  Indirect  Method  to  hybrid 
automata  were  proposed.  The  focus  our  work  is  on  input/output  notions  of  stability  that 
are  more  relevant  to  the  approximation  and  control  problems. 
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Since  our  interest  in  deterministic  finite  state  machines  stems  from  problems  of  approx¬ 
imation  and  robust  control,  we  would  like  to  call  a  nominal  system  ’stable’  if  its  output 
response  is  not  highly  sensitive  to  perturbations  (for  instance,  due  to  unmodeled  dynamics). 
Thus  we  explore  three  potentially  useful  notions  of  stability:  The  notion  of  input/output 
stability,  which  is  widely  used  in  classical  LTI  robust  control.  The  notion  of  incremental 
input/output  stability,  which  measures  the  sensitivity  of  the  system  to  perturbations  of  the 
input,  and  which  can  be  considered  to  be  a  viable  alternative  to  input/output  stability  when 
the  systems  in  question  are  nonlinear.  The  notion  of  external  stability,  which  measures  the 
sensitivity  of  the  output  of  a  system  to  changes  in  its  initial  conditions,  and  which  was 
introduced  in  [7]  and  shown  to  be  of  relevance  to  the  problem  of  approximating  an  analog 
state  system  with  finite  input  and  output  alphabets  by  a  deterministic  finite  state  machine. 

Summary:  We  consider  three  notions  of  stability,  input/output  stability,  external  sta¬ 
bility  and  incremental  input/output  stability,  as  they  apply  to  deterministic  finite  state 
machine  systems.  We  propose  LP  based  algorithms  for  verifying  stability  of  a  system,  or 
lack  thereof,  in  the  input /output  and  the  external  sense.  We  show  that  for  a  class  of  sys¬ 
tems,  incremental  I/O  stability  and  external  stability  are  equivalent  notions,  stronger  than 
the  notion  of  I/O  stability. 

1.1  A  New  Hybrid  Control  Architecture 

A  hybrid  control  structure  is  proposed  which  employs  a  finite  state  automaton  (FSA)  as  well 
as  a  continuous-time  feedback  structure  to  be  used  for  control  of  continuous-time  plants. 
The  motivation  for  exploring  such  a  structure  stems  from  real-world  implementation  issues 
that  exist  with  current  design  methodologies  for  either  strictly  analog  or  strictly  digital 
control.  The  main  goal  of  this  work  is  to  investigate  the  utility  of  such  a  structure.  First,  we 
wish  to  determine  whether  the  set  of  systems  for  which  the  given  hybrid  feedback  structure 
can  be  designed  is  sufficiently  broad,  and  we  wish  to  find  an  efficient  set  of  tools  which  can 
be  used  to  analyze  and  design  such  feedback  structures.  Second,  for  the  class  of  systems 
for  which  the  structure  is  known  to  be  stabilizing,  we  wish  to  compare  its  performance  to 
already  existing  feedback  structures.  In  particular,  we  wish  to  compare  performance  on 
measures  such  as  system  gain  bounds  (£2,  £00,  etc.),  memory  requirements,  settling  time, 
disturbance  rejection,  etc.. 

Much  attention  over  the  past  year  has  been  given  to  the  specific  case  where  the  continuous¬ 
time  plant  is  an  unstable  second  order  LTI  system,  and  where  the  continuous-time  portion 
of  the  feedback  structure  is  a  set  of  proportional  gains.  For  this  particular  case,  necessary 
and  sufficient  conditions  for  stabilizability  of  a  given  unstable  plant  have  been  discovered 
for  a  relaxation  of  the  given  problem  in  which  the  FSA  has  access  to  the  full  state  of  the 
plant.  Moreover,  the  stabilizability  of  a  given  second  order  LTI  plant  can  be  characterized 
in  terms  of  the  classical  control  notion  of  root  locus. 

The  goal  of  more  recent  work  has  been  to  investigate  the  stability  and  £00  gain- 
boundedness  properties  for  second  order  LTI  systems  in  which  the  finite  state  automaton 
has  access  to  only  a  single  output  of  the  plant  (rather  than  the  full  state  vector) .  Standard 
Lyapunov/storage  function  methods  for  even  simple  examples  have  proved  to  be  very  diffi¬ 
cult,  so  other  methods  of  analysis  have  been  under  consideration.  One  such  method  which 
uses  the  fact  that  the  system  trajectories  return  to  a  particular  region  of  the  state  space 


infinitely  often  has  proved  successful  analysis-wise  for  a  specific  example.  Investigation  of 
this  technique  as  a  means  of  proving  Coo  gain  boundedness  for  a  general  second  order  LTI 
plant  has  also  been  obtained.  The  results  are  documented  in  the  conference  papers  [1,  2] 
which  are  under  preparation  for  journal  publication. 

1.2  Model  Reduction  of  Hidden  Markov  Models 

Hidden  Markov  Models  have  been  used  extensively  in  many  fields  of  engineering  and  science 
in  order  to  model  large  scale  complex  stochastic  processes  that  evolve  on  concealed  finite 
state  spaces.  The  main  tasks  of  statistical  inference,  forecasting,  and  decision  making 
performed  by  means  of  mathematical  abstractions  of  real  world  applications  are  to  a  large 
extend  hindered  by  computational  intractability  issues  associated  with  the  large  state  space 
order  of  the  underlying  stochastic  process.  This  situation  is  reflected  by  the  commonly  used 
term  “curse  of  dimensionality” . 

The  goal  of  this  research  is  to  alleviate  that  problem,  by  developing  computational  tools  that 
perform  model  reduction  of  Hidden  Markov  Models.  The  main  aspects  of  this  work  consist 
of  defining  appropriate  measures  of  complexity,  identifying  relevant  approximation  metrics 
for  control  purposes,  deriving  reduced  models  by  means  of  efficient  algorithms  and  providing 
a  priori  error  bounds  that  validate  the  low  order  dynamic  systems.  The  computation  of  low 
complexity  approximations  of  Hidden  Markov  Models  also  constitutes  an  integral  part  of  a 
robust  control  framework  for  systems  whose  dynamics  are  governed  by  probabilistic  laws, 
as  well  as  nonlinear  deterministic  and  hybrid  systems. 

Progress  has  been  made  by  developing  such  tools  for  a  related  class  of  systems,  namely 
discrete-time  linear  systems  with  white  noise  coefficients.  This  work  has  been  summarized 
in  [8].  In  this  work,  we  have  shown  how  the  Balanced-truncation  algorithm,  popular  for  LTI 
systems,  can  be  extended  to  provide  a  reduced  order  model  with  guaranteed  error  bounds. 
The  construction  of  the  balancing  transformation  is  obtained  by  solving  LMIs.  Current 
research  is  focused  on  mapping  these  techniques  to  the  Hidden  Markov  Model  case  and  is 
expected  to  be  completed  sucessfully  in  the  following  months. 

1.3  Fundamental  Limitations  of  Performance  in  the  Presence  of  Finite 
Capacity  Feedback 

Motivated  by  applications,  such  as  remote  feedback,  control  in  the  presence  of  information 
constraints  has  received  considerable  attention.  The  exploration  of  such  problems  is  exciting 
as  they  foster  the  interaction  between  the  disciplines  of  Information  Theory  and  Control. 
So  far,  research  in  this  field  has,  primarily,  directed  its  attention  to  stabilization. 

The  basic  framework  comprises  a  plant,  a  channel,  an  encoder  and  a  decoder,  which  im¬ 
plicitly  embeds  a  controller.  Measurements  of  the  plant’s  output  must  be  encoded  and  sent 
through  the  channel.  The  information,  received  at  the  other  end  of  the  channel,  is  decoded 
and  used  to  generate  a  control  signal. 

We  have  investigated  the  fundamental  limitations  of  performance  arising  from  finite  capac¬ 
ity  feedback,  operating  on  arbitrary  alphabets.  Using  Information  theoretic  principles,  we 
have  derived  fundamental  relations  between  key  quantities.  In  particular,  under  stationarity 


assumptions,  we  show  that  the  information  rate  at  the  channel,  denoted  as  I,  must  satisfy: 

1  r 

I  >  —  max{0,  log  S}dw  +  fy 
27r  J 

where  S  is  the  closed  loop  sensitivity  and  Ig.  is  the  information  rate  required  for  stabilization. 
As  such,  this  inequality  indicates  that  /  —  fy  determines  a  fundamental  limit  on  disturbance 
attenuation. 

Using  the  same  techniques,  we  have  proven  that  Bode’s  integral  formula  holds  in  the  pres¬ 
ence  of  feedback  over  arbitrary  alphabets.  Both  fundamental  limitations  must  be  always 
satisfied  since  they  originate  from  distinct  constraints:  Bode’s  limitation  results  from  causal¬ 
ity,  while  the  above  limitation  is  associated  with  information  rate  constraints.  These  results 
are  documented  in  [9,  10,  14]. 

1.3.1  Vehicle  Routing:  Dynamic  Pick-up  and  Delivery  Problem 

Our  research  on  the  problem  of  control  with  communication  constraints  reported  in  [9, 
10,  14]  expanded  to  address  problems  concerning  larger  networks  of  systems.  In  here,  we 
briefly  describe  our  work  on  the  control  of  networked  mobile  systems.  Since  many  such 
formulations  can  result  in  intractable  problems,  a  major  objective  of  this  project  is  to  find 
abstracted  models  that  capture  some  of  the  important  tradeoffs  in  performance  objectives 
and  constraints  which  are  also  tractable  in  terms  analysis  and  optimization.  An  example  of 
such  a  simplified  model  of  networked  agents  is  the  Dynamic  Pickup  and  Delivery  Problem 
(DPDP)  described  below  as  an  example  of  vehicle  routing  problem,  and  is  reported  in 
[11,  12,  13]. 

Vehicle  Dynamics:  Assume  there  are  n  vehicles  in  a  geographic  area  .4  C  R^,  which 
is  a  convex,  compact  set  with  volume  A.  For  simplicity  of  this  exposition,  we  consider 
A  =  [0,VA]'^.  Each  vehicle  may  move  in  any  direction  at  any  time  with  a  velocity  of 
magnitude  <  v.  In  the  simplest  case,  v  does  not  depend  on  n,  however,  if  we  want  to 
consider  congestion,  we  may  allow  the  velocity  to  decrease  as  n  increases. 

Service  Requests  Messages  (service  requests)  are  generated  according  to  a  Poisson  process 
with  time  intensity  A(n).  We  assume  that  the  rate  of  messages  arriving  to  the  system 
increases  much  faster  than  the  number  of  vehicles  in  the  system  available  to  service  these 
messages.  Associated  with  each  message  j  are  source  and  destination  locations  denoted  by 
s{j)  6  A  and  d{j)  €  A  respectively.  Source  locations  are  independently  and  identically 
distributed  (IID)  in  A  according  to  the  distribution  <j)s  :  A  ^  R"*".  Similarly,  destination 
locations  are  IID  with  distribution  cpd'  A— ^  R"*". 

Objective  The  messages  need  to  be  picked  up  from  their  source  locations  and  delivered 
to  their  destination  locations  by  the  vehicles.  A  message  is  picked  up  (delivered)  when  a 
vehicle  spends  a  fixed  on-site  service  time  of  s(n)  at  the  source  (delivery)  location  to  pick 
up  (deliver)  the  message. 

Control  Policies 

A  control  policy,  tt,  is  a  set  of  decision  making  rules  that  decides  the  pickup  and  delivery 
schedule  of  arriving  messages,  based  on  a  set  of  constraints  on  the  information  available 


to  the  vehicle.  In  this  paper,  we  consider  policies  tt  =  {t^AiT^s)  that  can  be  decomposed 
into  two  components,  assignment  and  service.  An  assignment  policy,  tt^,  describes  how  a 
centralized  controller  assigns  arriving  messages  to  vehicles  on  a  real-time  basis.  A  service 
policy,  TTs,  describes  how  each  vehicle  performs  the  pickup  and  delivery  of  its  assigned 
messages.  We  assume  that  neither  the  vehicles  nor  the  centralized  assignment  controller 
have  any  knowledge  of  individual  messages  before  they  arrive  although  the  overall  message 
arrival  process  and  source  and  destination  distributions  are  known. 

Information  Structure  We  further  restrict  the  set  of  assignment  policies  according  to  the 
information  available  to  the  controller  in  making  message  assignments.  For  example,  we  can 
consider  two  types  of  information  structures.  The  first  is  the  Source  only  information.  In 
such  a  case,  when  a  message  arrives,  its  source  location  is  known  to  the  centralized  controller, 
which  is  consequently  communicated  to  the  appropriate  vehicle.  Vehicles,  however,  do  not 
know  the  destination  of  messages  until  they  pick  them  up.  The  second  is  the  Source- 
destination  information.  In  this  case,  when  a  message  arrives,  both  its  source  and  also 
its  destination  location  are  known  to  the  centralized  controller  and  both  are  consequently 
communicated  to  the  appropriate  vehicle. 

Performance  Metrics:  One  metric  that  has  received  much  attention  in  queuing  theory  is 
the  average  delay.  The  delay  of  message  j,  denoted  W{j),  is  defined  to  be  the  elapsed  time 
between  the  message’s  arrival  to  the  system  and  its  delivery  to  its  destination  location.  This 
includes  any  time  the  message  waits  to  be  picked  up,  the  onsite  service  time  for  pickup, 
travel  time  on  the  vehicle  before  arriving  at  the  delivery  location,  and  finally  onsite  service 
time  for  delivery.  The  quantity  W  is  defined  to  be 

W  =  limsupE[IF(j)].  (1) 

j—*oo 

W  is  the  steady  state  expected  value  of  the  delay  of  a  message  in  the  system.  We  say  that 
the  system  is  stable  if  IF  <  oo.  If  tt  is  a  stable  policy,  IF(7r)  is  defined  to  be  the  limiting 
expected  delay  associated  with  that  policy.  Note  that  W{Tr)  =  W (tt^  ,  Trg)  is  a  function  of 
both  the  assignment  policy  and  the  service  policy. 

Problem  Definition:  Given  the  problem  defined  above,  we  seek  stable  policies  that  mini¬ 
mize  that  average  delay  per  message.  Equivalently,  we  must  solve  the  following  optimization 
problem; 

min  W  (tt)  tt  is  a  stabilizing  policy 

(’r/i,7rs) 

This  problem  minimizes  the  weighted  sum  of  average  delays  over  all  of  the  vehicles  ac¬ 
cording  to  a  joint  constraint  on  the  policies  used  by  the  individual  vehicles.  If  the  minimum 
is  finite,  each  of  the  Wi{n)  must  be  finite  as  well  and  the  system  is  stable. 

We  call  this  problem  the  Dynamic  Pickup-Delivery  Problem  (DPDP).  It  is  important 
to  note  that  OPTl  is  still  a  difficult  optimization  problem.  However,  we  will  discuss  how 
the  complexity  of  this  problem  can  be  substantially  reduced  by  considering  the  behavior  of 
this  problem  for  large  n. 

Below  we  summarize  the  results  for  the  Source-only  information  structure. 


Theorem  1 


(a)  For  any  policy  in  IIso  under  the  Source  Only  information  structure,  the  average  delay 
per  message  scales  as 


(  Kn)A  \ 


n(l  -  2p) 
2A(n) 


(2) 


with  constant  7  =  2/3v^^.  For  A(n)  =  Q,{n)  this  bound  scales  as  Wso  —  fi(A(n)j4/u^(l 
pfn). 

(b)  Further,  there  exists  a  policy  using  Source  Only  information,  for  which  the  average 
delay  scales  as  0{\{n)A/v^{\  —  p)‘^n)  for  all  A(n).  Therefore  the  lower  bound  scaling 
is  achievable. 


Similarly,  we  summarize  the  results  for  the  Source-Destination  information  structure. 

Theorem  2 


(a)  For  any  policy  in  fl^r)  under  the  Source-Destination  information  structure,  the  average 
delay  per  message  with  A(n)  =  Q(n)  scales  as 


WsD  > 


7^  X{n)A 
4  ^2(1  —  pYrfi!'^ 


with  constant  7  =  2/3%/^. 


(3) 


(b)  Further,  there  exists  a  policy  using  Source  and  Destination  information  for  which  the 
delay  scales  as  0{\{n)A/v^{\.  —  p^vA/"^)  for  A(n)  =  Therefore  the  lower 

bound  scaling  is  achievable  in  heavy  traffic. 
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